%
% NOTES:  
%  - Be sure to place captions BEFORE labels in figures and tables!
%    Otherwise, numbering will be incorrect.  For example, use the following:
%       \caption{TAO Info}
%       \label{fig:taoinfo}
%  - Use \break to indicate a line break (needed to prevent long strings in
%    \tt mode from running of the page)
%
% ---------------------------------------------------------------

\chapter{Using TAO Solvers}
%\sindex{continuous optimization}
\label{chapter:tao_solver}

TAO contains unconstrained minimization, bound-constrained minimization, 
nonlinear complementarity, nonlinear least squares solvers, and solvers
for optimization problems with partial differential equation constraints.
The structure of these problems can differ significantly, but TAO has a 
similar interface to all its solvers.  
Routines that most solvers have in common are discussed in 
this chapter.
A complete list of options can be found by consulting the manual pages.
Many of the options can also be set at the command line.  These options
can also be found by
running a program with the {\tt -help} option.

\section{Header File}

TAO applications written in C/C++ should have the statement 
\begin{verbatim}
   #include <petsctao.h>
\end{verbatim}
\noindent
in each file that uses a routine in the TAO libraries.


\section{Creation and Destruction}

A TAO solver can be created by calling the
\findex{TaoCreate()}
\begin{verbatim}
   TaoCreate(MPI_Comm comm,Tao *newsolver);
\end{verbatim}
routine. 
Much like creating PETSc vector and matrix objects, 
the first argument is an MPI {\em communicator}.
An MPI \cite{using-mpi} communicator
indicates a collection of processors that will be used to evaluate the
objective function, compute constraints, and provide derivative information.
When only one processor is being used, the communicator {\tt PETSC\_COMM\_SELF}
can be used with no understanding of MPI.
Even parallel users need to be familiar with only the basic concepts 
of message passing and  distributed-memory computing. 
Most applications running TAO in
parallel environments can employ the communicator {\tt
PETSC\_COMM\_WORLD} to indicate all processes known to PETSc in a given run.

The routine
\findex{TaoSetType()}
\begin{verbatim}
   TaoSetType(Tao tao,TaoType type);
\end{verbatim}
\noindent
can be used to set the algorithm TAO uses to solve the application.
The various types of TAO solvers and the flags that identify them 
will be discussed in the following chapters.
The solution method should be carefully chosen depending on
the problem being solved.  Some solvers, for instance, are meant for
problems with no constraints, whereas other solvers acknowledge constraints
in the problem and handle them accordingly.
The user must also be aware of the derivative information that is available.
Some solvers require second-order information, while other solvers require
only gradient or function information.
The command line option \texttt{-tao\_method} (or equivalently 
\texttt{-tao\_type}) followed by a TAO method
will override any method specified by the second argument.
The command line option {\tt -tao\_method tao\_lmvm}, for instance,
will specify the limited-memory, variable metric method for unconstrained
optimization.  Note that the {\tt TaoType} variable is a string that requires
quotation marks in an application program, but quotation marks are not required
at the command line.

Each TAO solver that has been created should also be destroyed by using
the  
\findex{TaoDestroy()}
\begin{verbatim}
   TaoDestroy(Tao tao);
\end{verbatim}
command. 
This routine frees the internal data structures used by the solver.


\section{TAO Applications}
\sindex{application}
\label{section:taoapplication}
\label{section:petscapp}

The solvers in TAO address applications that have a set of variables, an objective
function, and possibly constraints on the variables.  Many solvers also
require derivatives
of the objective and constraint functions.
To use the TAO solvers, the application developer must 
define a set of variables, implement routines that evaluate the 
objective function and constraint functions, and pass this information
to a TAO application object.   

TAO uses vector and matrix objects to pass this information from the
application to the solver.   The set of variables, for instance, is
represented in a vector.
The gradient of an objective function $f: \, \Re^n \to \Re$,
evaluated at a point, is also represented as a vector.
Matrices,  on the other hand,
can be used to represent the Hessian of $f$ or the Jacobian of a constraint
function $c: \, \Re^n \to \Re^m$.  The TAO solvers use
these objects to compute a solution to the application.

\subsection{Defining Variables}
In all the optimization solvers, the application must provide
a {\bf Vec} object of appropriate dimension to represent the variables.
This vector will be cloned by the solvers to create additional work
space within the solver.
If this vector is distributed over multiple processors, it
should have a parallel distribution that allows
for efficient scaling, inner products, and
function evaluations.  This vector can be passed to the
application object by using the  \findex{TaoAppSetInitialSolutionVec()}
\begin{verbatim}
   TaoSetInitialVector(Tao,Vec);
\end{verbatim}
routine. 
When using this routine, the application should initialize the vector with
an approximate solution of the optimization problem before calling the
TAO solver.
This vector will be used by the TAO solver to store the solution.
Elsewhere in the application, 
this solution vector can be retrieved from the application object 
by using the  \findex{TaoGetSolutionVector()}
\begin{verbatim}
   TaoGetSolutionVector(Tao,Vec *);
\end{verbatim}
routine. 
This routine takes the address of a {\tt Vec} in the second argument and sets it to
the solution vector used in the application.

\subsection{Application Context}  \sindex{application context}
Writing a TAO application may require
use of an {\em application context}.
An application context is a structure or object defined by an
application developer, passed
into a routine also written by the application developer, 
and used within the routine to perform its stated task.
 
For example, a routine that evaluates an objective function may need
parameters, work vectors, and other information.   This information,
which may be specific to an application and necessary to evaluate the objective,
can be collected in a single structure and used as one of the
arguments in the routine.
The address of this structure will be cast as type {\tt (void*)} and passed to
the routine in the final argument.
Many examples of these structures are included in the
TAO distribution.

This technique offers several advantages.
In particular, it allows for a uniform interface between TAO and 
the applications.   The fundamental information needed by TAO 
appears in the arguments of the routine, while data specific to an application
and its implementation is confined to an opaque pointer.
The routines can access information created outside the 
local scope without the use of global variables.
The TAO solvers and application objects will never access this structure, 
so the application developer has complete freedom to define it. If no 
such structure or needed by the application then a NULL pointer can be used.



\subsection{Objective Function and Gradient Routines}\label{sec:fghj}

TAO solvers that minimize an objective function require
the application to evaluate the objective function.  Some solvers
may also require the application to evaluate
derivatives of the objective function.  
Routines that perform these computations must be identified
to the application object and must follow a strict calling sequence.

Routines should follow the form
\begin{verbatim}
   PetscErrorCode EvaluateObjective(Tao,Vec,PetscReal*,void*);
\end{verbatim}
in order to evaluate an objective function $f: \, \Re^n \to \Re$. 
The first argument is the TAO Solver object, the second argument is the
$n$-dimensional vector that identifies where the objective should be evaluated, 
and the fourth argument is an application context.
This routine should use the third argument to return the objective value 
evaluated at the point
specified by the vector in the second argument.

This routine, and the application context, should be passed to the 
application object by using
the  \findex{TaoSetObjectiveRoutine()}
\begin{verbatim}
   TaoSetObjectiveRoutine(Tao,
                     PetscErrorCode(*)(Tao,Vec,PetscReal*,void*),
                     void*);
\end{verbatim}
routine. 
The first argument in this routine is the TAO solver object, 
the second argument is a function pointer to the routine that 
evaluates the objective, and the third
argument is the pointer to an appropriate application context.  
Although the final argument may point to anything, it must be cast as a {\tt (void*)} type.
This pointer will be passed back to the developer in the fourth argument of the
routine that evaluates the objective.  In this routine, the pointer can be cast
back to the appropriate type.  Examples of these structures and their 
usage are provided in the distribution.

Many TAO solvers also require gradient information from the 
application \sindex{gradients}.
  The gradient of the objective function is specified in a similar manner.
Routines that evaluate the gradient should have the calling sequence
\begin{verbatim}
   PetscErrorCode EvaluateGradient(Tao,Vec,Vec,void*);
\end{verbatim}
\noindent
where the first
argument is the TAO solver object, the second argument is the variable
vector, the third argument is the gradient vector, and the fourth argument is
the user-defined application context.  Only the third argument in this
routine is different from the arguments in the routine for evaluating
the objective function.  The numbers in the gradient vector have no
meaning when passed into this routine, but they should represent the gradient
of the objective at the specified point at the end of the routine.
This routine, and the user-defined pointer, can be passed to the application
object by using the  \findex{TaoSetGradientRoutine()}
\begin{verbatim}
   TaoSetGradientRoutine(Tao,
                     PetscErrorCode (*)(Tao,Vec,Vec,void*),
                     void *);
\end{verbatim}
routine. 
In this routine, the first argument is the Tao object, the second argument
is the function pointer, and the third object is the application context, cast
to {\tt (void*)}.

Instead of evaluating the objective and its gradient in separate
routines, TAO also allows the user to evaluate the function and the gradient
in the same routine.  In fact, some solvers are more efficient when
both function and gradient information can be computed in the same routine.
These routines should follow the form
\begin{verbatim}
   PetscErrorCode EvaluateFunctionAndGradient(Tao,Vec,
                     PetscReal*,Vec,void*);
\end{verbatim}
\noindent
where the first
argument is the TAO solver and the second
argument points to the input vector for use in evaluating the
function and gradient. The third argument should return the
function value, while the fourth argument should return the gradient vector.
The fifth argument is a pointer to a user-defined context.
This context and the name of the routine should be set with the
call \findex{TaoSetObjectiveAndGradientRoutine()}
\begin{verbatim}
   TaoSetObjectiveAndGradientRoutine(Tao,
                     PetscErrorCode (*)(Tao,Vec,PetscReal*,Vec,void*),
                     void *);
\end{verbatim}
where the arguments are the TAO application, a
function name, and a pointer to a user-defined context.


The TAO example problems demonstrate the use of these application contexts
as well as specific instances of function, gradient, and Hessian 
evaluation routines.
All these routines should return the integer $0$ after 
successful completion and a nonzero integer if the function
is undefined at that point or an error occurred.

\subsection{Hessian Evaluation}
\label{sec:matrixfree}
\label{sec:finitedifference}

Some optimization routines also require a Hessian matrix from the user.
The routine that evaluates the Hessian should have the form 
\begin{verbatim}
   PetscErrorCode EvaluateHessian(Tao,Vec,Mat,Mat,void*);
\end{verbatim}
where the first argument of this routine is a TAO solver object.  The
second
argument is the point at which the Hessian should be evaluated.  The
third argument is the Hessian matrix, and the sixth argument is a
user-defined context.
Since the Hessian matrix is usually used in solving
a system of linear equations, a preconditioner for the matrix is often
needed.  The fourth argument is the matrix that will be used
for preconditioning the linear system; in most cases, this
matrix will be the same as the Hessian matrix.  The fifth
argument is the flag used to set the Hessian matrix and
linear solver in the routine {\tt KSPSetOperators()}.

One can set the Hessian evaluation routine by calling the \sindex{Hessian}
\findex{TaoSetHessianRoutine()}
\begin{verbatim}
   TaoSetHessianRoutine(Tao,Mat H, Mat Hpre,
                     PetscErrorCode (*)(Tao,Vec,Mat,Mat,
                     void*), void *);
\end{verbatim}
routine. 
The first argument is the TAO Solver object. The second and third arguments
are, respectively, the Mat object where the Hessian will be stored and 
the Mat object
that will be used for the preconditioning (they may be the same). The fourth 
argument is the function that evaluates the Hessian, 
and the fifth argument is a pointer to a user-defined context,
cast to {\tt (void*)}.

\subsubsection{Finite Differences} \sindex{finite differences}
Finite-difference approximations can be used to compute the gradient and the
Hessian of an objective
function.  These approximations will slow the solve considerably and 
are recommended primarily  
for checking the accuracy of hand-coded gradients and Hessians.
These routines are
\findex{TaoDefaultComputeGradient()}
\begin{verbatim}
   TaoDefaultComputeGradient(Tao, Vec, Vec, void*);
\end{verbatim}
and 
\findex{TaoDefaultComputeHessian()}
\begin{verbatim}
   TaoDefaultComputeHessian(Tao, Vec, Mat*, Mat*,void*);
\end{verbatim}
respectively. They can be set by using {\tt
TaoSetGradientRoutine()} and 
{\tt TaoSetHessianRoutine()} or through the options database with the
options {\tt -tao\_fdgrad} and {\tt -tao\_fd}, respectively.

The efficiency of the finite-difference Hessian can be improved if the
coloring of the matrix is known.  If the application programmer creates
a PETSc {\tt MatFDColoring} object, it can be applied to the finite-difference
approximation by setting the Hessian evaluation routine to
\begin{verbatim}
   TaoDefaultComputeHessianColor(Tao, Vec, Mat*, Mat*,void* );
\end{verbatim}
and using the {\tt MatFDColoring} object as
the last ({\tt void *}) argument to {\tt TaoSetHessianRoutine()}.

One also can use finite-difference approximations to directly check
the correctness of the gradient and/or Hessian evaluation routines.
This process can be initiated from the command line by using the special 
TAO solver 
{\tt tao\_fd\_test} together with the option
{\tt -tao\_test\_gradient} or {\tt -tao\_test\_hessian}.

\subsubsection{Matrix-Free Methods}
TAO fully supports matrix-free methods. The matrices specified in the
Hessian evaluation routine need not be conventional
matrices; instead, they can point to the data required to implement a
particular matrix-free method.  The matrix-free variant is allowed
{\em only} when the linear systems are solved by an iterative method
in combination with no preconditioning ({\tt PCNONE} or {\tt -pc\_type none}),
a user-provided preconditioner matrix, or a user-provided preconditioner
shell ({\tt PCSHELL}). In other words,
matrix-free methods cannot be used if a direct solver is to 
be employed. \sindex{matrix-free options} %\findex{PCSHELL}
Details about using matrix-free methods are provided in the
PETSc users manual \cite{petsc-user-ref}.


\subsection{Bounds on Variables}\label{sec:bounds}

Some optimization problems also impose constraints on the variables.
The constraints may impose simple bounds on the variables or
require that the variables satisfy a set of linear or  nonlinear equations.

The simplest type of constraint on an optimization problem puts lower
or upper bounds on the variables. 
Vectors that represent lower and upper bounds for each variable 
can be set with the   \sindex{bounds}\findex{TaoSetVariableBounds}
\begin{verbatim}
   TaoSetVariableBounds(Tao,Vec,Vec);
\end{verbatim}
command. 
The first vector and second vector should contain the lower and upper 
bounds, respectively.
When no upper or lower bound exists for a variable, the bound
may be set to {\tt TAO\_INFINITY} or {\tt TAO\_NINFINITY}.
After the two bound vectors have been set, they may be accessed
with the command  {\tt TaoGetVariableBounds()}.

Alternatively, it may be more convenient for the user to designate a routine 
for computing these bounds
that the solver will call before starting its algorithm.  This routine will
have the form
\begin{verbatim}
   PetscErrorCode EvaluateBounds(Tao,Vec,Vec,void*);
\end{verbatim}
where the two vectors, representing the lower and upper bounds respectfully, 
will be computed.

This routine can be set with the 
\begin{verbatim}
   TaoSetVariableBoundsRoutine(Tao
                     PetscErrorCode (*)(Tao,Vec,Vec,void*),void*);
\end{verbatim}
command.
   
Since not all solvers recognize the presence of bound constraints on 
variables, the user must be careful 
to select a solver that acknowledges these bounds.


\section{Solving}
Once the application and solver have been set up, the solver can be 
\findex{TaoSolve()} called by using the 
\begin{verbatim}
   TaoSolve(Tao);
\end{verbatim}
routine.
%This routine will call the TAO solver. 
We discuss several universal options below. 

\subsection{Convergence}\label{sec:customize}

Although TAO and its solvers set default parameters that are useful
for many problems, the user may need to modify these
parameters in order to change the behavior and convergence of various algorithms.

One convergence criterion for most algorithms concerns the number
of digits of accuracy needed in the solution.  In particular,
the convergence test employed by TAO attempts to stop when
the error in the constraints is less than $\epsilon_{crtol}$
and either
\[
\begin{array}{lcl}
||g(X)|| &\leq& \epsilon_{gatol}, \\
||g(X)||/|f(X)| &\leq& \epsilon_{grtol}, \quad \mbox{or} \\
||g(X)||/|g(X_0)| &\leq& \epsilon_{gttol},
\end{array}
\]
where $X$ is the current approximation to the true solution $X^*$
and $X_0$ is the initial guess.
$X^*$ is unknown, so TAO estimates $f(X) - f(X^*)$ with either 
the square of the norm of the gradient or the duality gap.
A relative tolerance of $\epsilon_{frtol}=0.01$ indicates that two
significant digits are desired in the objective function.
Each solver sets its own  convergence tolerances, but they can
be changed by using the routine
\findex{TaoSetTolerances()} 
\sindex{convergence tests}
{\tt TaoSetTolerances()}.
Another set of convergence tolerances 
terminates the solver when the norm of the gradient function
(or Lagrangian function for bound-constrained problems)
is sufficiently close to zero.

Other stopping criteria include a minimum trust-region radius or 
a maximum number of iterations.  These parameters can be set with
the routines {\tt Tao\-Set\-Trust\-Region\-Tolerance()}\sindex{trust region}\findex{TaoSetTrustRegionTolerance}
and {\tt Tao\-Set\-Max\-imum\-Iter\-ations()}\findex{TaoSetMaximumIterations()}.
Similarly, a maximum number of function evaluations can be set 
with the command 
\findex{TaoSetMaximumFunctionEvaluations()}
{\tt Tao\-Set\-Max\-imum\-Func\-tion\-Evaluations()}.
\texttt{-tao\_max\_it}, and \texttt{-tao\_max\_funcs}.

\subsection{Viewing Status}

To see parameters and performance statistics for the solver, the
routine
\begin{verbatim}
   TaoView(Tao tao)
\end{verbatim}
can be used.  This routine will display to standard output the number
of function evaluations need by the solver and other information
specific to the solver.  This same output can be produced by using the 
command line option {\tt -tao\_view}.

The progress of the optimization solver can be monitored with
the runtime option {\tt -tao\_monitor}.  Although monitoring routines
can be customized, the default monitoring routine will print out 
several relevant statistics to the screen.

The user also has access to information about the current solution.
The current iteration number, objective function value, gradient
norm, infeasibility norm, and step length 
can be retrieved with the follwing command.
\findex{TaoGetSolutionStatus()}
\begin{verbatim}
   TaoGetSolutionStatus(Tao tao, PetscInt *iterate, PetscReal *f,
                     PetscReal *gnorm, PetscReal *cnorm, PetscReal *xdiff,
                     TaoConvergedReason *reason)
\end{verbatim}
The last argument returns a code that indicates the reason that the solver 
terminated.  Positive 
numbers indicate that a solution has been found, while negative numbers
indicate a failure.  A list of reasons can be found in the manual page
for {\tt Tao\-Get\-Converged\-Reason()}.

\subsection{Obtaining a Solution}

After exiting the {\tt Tao\-Solve()} function, the solution, gradient, and 
\findex{TaoGetSolution()} dual variables (if available) can be recovered 
with the following routines.
\begin{verbatim}
   TaoGetSolutionVector(Tao, Vec *X);
   TaoGetGradientVector(Tao, Vec *G);
   TaoComputeDualVariables(Tao, Vec X, Vec Duals);
\end{verbatim}
Note that the {\tt Vec} returned by {\tt TaoGetSolutionVector} will be
the same vector passed to {\tt TaoSetInitialVector}.  This information 
can be obtained during user-defined routines such as a function evaluation 
and customized monitoring routine or after the solver has terminated.




\subsection{Additional Options}
Additional options for the TAO solver 
can be be set from the command line by using the  \sindex{options}
\findex{TaoSetOptions()}
\begin{verbatim}
   TaoSetFromOptions(Tao)
\end{verbatim}
routine. 
This command also provides information about runtime options when the
user includes the {\tt -help } option on the command line.

\section{Special Problem Structures}
Below we discuss how to exploit the special structures for three classes
of problems that TAO solves. 

\subsection{PDE-Constrained Optimization}\label{sec:pde_applications}
TAO can solve PDE-constrained optimization applications 
of the form
\[
\begin{array}{ll}
\displaystyle \min_{u,v} & f(u,v) \\
\mbox{subject to} & g(u,v) = 0,
\end{array}
\]
where the state variable $u$ is the solution to the discretized partial 
differential equation defined by $g$ and parametrized by the design 
variable $v$, and $f$ is an objective function.  In this case, the 
user needs to set routines for computing the objective function
and its gradient, the constraints, and the Jacobian of the constraints
with respect to the state and design variables.  TAO also needs to know 
which variables in the solution vector correspond to state variables 
and which to design variables.

The objective and gradient routines are set as for other TAO applications,
with {\tt Tao\-Set\-Object\-ive\-Routine()} and {\tt
Tao\-Set\-Gradient\-Routine()}.  The user can also provide a fused
objective function and gradient
evaluation with {\tt Tao\-Set\-Objective\-And\-Gradient\-Routine()}.
The input and output vectors include the combined state and design 
variables.  Index sets for the state and design variables must be 
passed to TAO by using the function
\begin{verbatim}
   TaoSetStateDesignIS(Tao, IS, IS);
\end{verbatim}
where the first IS is a PETSc Index\-Set containing the indices of the
state variables and the second IS the design variables.

Nonlinear equation constraints have the general form $c(x) = 0$, where 
$c: \Re^n \to \Re^m$.  These constraints should be specified in a routine, 
written by the user, that evaluates $c(x)$.  The routine that evaluates
the 
constraint equations should have the form
\begin{verbatim}
   PetscErrorCode EvaluateConstraints(Tao,Vec,Vec,void*);
\end{verbatim}
The first argument of this routine is a TAO solver object.  The
second argument is the variable vector at which the constraint function
should be evaluated.  
The third argument is the vector of function values $c(x)$, and the fourth
argument is a pointer to a user-defined context.  This routine and the 
user-defined context should be set in the TAO solver with the 
\findex{TaoSetConstraintsRoutine()}
\begin{verbatim}
   TaoSetConstraintsRoutine(Tao,Vec,
                     PetscErrorCode (*)(Tao,Vec,Vec,void*),
                     void*);
\end{verbatim}
command. 
In this function, the first argument is the TAO solver object, the second 
argument a vector in which to store the constraints, the third argument is
a function point to the routine for evaluating the constraints, and the
fourth argument is a pointer to a user-defined context.

The Jacobian of $c(x)$ is the matrix in $\Re^{m \times n}$ such that each 
column contains the partial derivatives of $c(x)$ with respect to one 
variable.  The evaluation of the Jacobian of $c$ should be performed 
by calling the 
\begin{verbatim}
   PetscErrorCode JacobianState(Tao,Vec,Mat,Mat,Mat,void*);
   PetscErrorCode JacobianDesign(Tao,Vec,Mat*,void*);
\end{verbatim}
routines. 
In these functions, The first arguemnt is the TAO solver object.  The second 
argument is the variable vector at which to 
evaluate the Jacobian matrix, the third argument is the Jacobian matrix,
and the last argument is a pointer to a user-defined context. The fourth
and
fifth arguments of the Jacobian evaluation with respect to the state
variables
are for providing PETSc matrix objects for the preconditioner and for
applying
the inverse of the state Jacobian, respectively.  This inverse matrix may be 
{\tt PETSC\_NULL}, in which case TAO will use a PETSc Krylov subspace 
solver to solve the state system.  These evaluation routines should be 
registered 
with TAO by using the 
\begin{verbatim}
   TaoSetJacobianStateRoutine(Tao,Mat,Mat,Mat,
                     PetscErrorCode (*)(Tao,Vec,Mat,Mat,
                     void*), void*);
   TaoSetJacobianDesignRoutine(Tao,Mat,
                     PetscErrorCode (*)(Tao,Vec,Mat*,void*), 
                     void*);
\end{verbatim}
routines. 
The first argument is the TAO solver object, and the second argument is the
matrix 
in which the Jacobian information can be stored.  For the state Jacobian,
the 
third argument is the matrix that will be used for preconditioning, and the 
fourth argument is an optional matrix for the inverse of the state
Jacobian.
One can use {\tt PETSC\_NULL} for this inverse argument and let PETSc
apply 
the inverse using a KSP method, but faster results may be obtained by
manipulating the structure of the Jacobian and providing an inverse.
The fifth argument is the function pointer, and the sixth argument is
an optional user-defined context.  Since no solve is performed with the
design Jacobian, there is no need to provide preconditioner or inverse
matrices.


\subsection{Nonlinear Least Squares}\label{sec:evalsof}
For nonlinear least squares applications, we are solving
the optimization problem
\begin{equation*}
\min_{x} \;\frac{1}{2}||r(x)||_2^2.
\end{equation*}
For these problems, the objective function value should be computed as a 
vector of residuals, $r(x)$, computed with a function of the form
\begin{verbatim}
   PetscErrorCode EvaluateResidual(Tao,Vec,Vec,void*);
\end{verbatim}
and set with the
\begin{verbatim}
   TaoSetResidualRoutine(Tao, PetscErrorCode (*)(Tao,Vec,Vec,void*), 
                         void *);
\end{verbatim}
routine. If required by the algorithm, the Jacobian of the residual, $J = \partial r(x) / \partial x$, 
should be computed with a function of the form 
\begin{verbatim}
   PetscErrorCode EvaluateJacobian(Tao,Vec,Mat,void*);
\end{verbatim}
and set with the 
\begin{verbatim}
   TaoSetJacobianResidualRoutine(Tao, PetscErrorCode (*)(Tao,Vec,Mat,void*),
                                 void *);
\end{verbatim}
routine.

\begin{comment}
The computation of the Jacobian of the separable objective routine 
should be in a routine that looks like
\begin{verbatim}
   PetscErrorCode EvaluateJacobian(Tao,Vec,Mat,Mat,void*);
\end{verbatim}
This function can be registered with TAO by using the function
\begin{verbatim}
   TaoSetJacobianRoutine(Tao,Mat J, Mat Jpre,
                     PetscErrorCode (*)(Tao,Vec,Mat,Mat,
                     void*), void *);
\end{verbatim}
The first argument is the TAO solver object, the second and third arguments
are the Mat object where the Jacobian will be stored and the Mat object
that will be used for the preconditioning (they may be the same). The
fourth 
argument is the function that evaluates the Jacobian, 
and the fifth argument is a pointer to a user defined context,
cast as a {\tt void*} pointer.
\end{comment}



\subsection{Complementarity}
Complementarity applications have equality constraints in the form of 
nonlinear equations 
$C(X) = 0$, where $C: \Re^n \to \Re^m$.
These constraints should be specified in a 
routine written by the user with the form
\begin{verbatim}
   PetscErrorCode EqualityConstraints(Tao,Vec,Vec,void*);
\end{verbatim}
that evaluates {\tt C(X)}.
\noindent
The first argument of this routine is a TAO Solver object.  The second argument
is the variable vector $X$ at which the constraint function should be 
evaluated.  
The third argument is the output vector of function values $C(X)$, and the fourth
argument is a pointer to a user-defined context.

This routine and the user-defined context 
must be registered with TAO by using the
\findex{TaoSetConstraintsRoutine()}
\begin{verbatim}
   TaoSetConstraintRoutine(Tao, Vec,
                     PetscErrorCode (*)(Tao,Vec,Vec,void*),
                     void*);
\end{verbatim}
command. 
\noindent
In this command, the first argument is TAO Solver object,
the second argument is vector in which to store the function values,
the third argument is the user-defined routine that evaluates $C(X)$,
and the fourth argument is a pointer to a user-defined context that will
be passed back to the user.

The Jacobian of the function is the matrix in $\Re^{m \times n}$
such that each column contains the partial derivatives of {\tt f} with respect
to one variable. 
The evaluation of the Jacobian of $C$ should be performed in a routine
of the form
\begin{verbatim}
   PetscErrorCode EvaluateJacobian(Tao,Vec,Mat,Mat,void*);
\end{verbatim}
\noindent
In this function, the first argument is the TAO Solver object and the 
second argument is the variable vector at which to evaluate the 
Jacobian matrix. The third argument is the Jacobian matrix,
and the sixth argument is a pointer to a user-defined context.
Since the Jacobian matrix may be used in solving a system of linear equations,
a preconditioner for the matrix may be needed.  The fourth argument is the matrix that will be used
for preconditioning the linear system; in most cases, this
matrix will be the same as the Hessian matrix.  The fifth
argument is the flag used to set the Jacobian matrix and
linear solver in the routine {\tt KSPSetOperators()}.

This routine should be specified to TAO by using the
\findex{TaoAppSetJacobianRoutine()}
\begin{verbatim}
   TaoSetJacobianRoutine(Tao,Mat J, Mat Jpre,
                     PetscErrorCode (*)(Tao,Vec,Mat,Mat, 
                     void*), void*);
\end{verbatim}
\noindent
command.
The first argument is the TAO Solver object; the second
and third arguments are the Mat objects in which the Jacobian will be stored
and the Mat object that will be used for the preconditioning (they may
be the same), respectively. 
The fourth argument is the function pointer; and the fifth argument is
an optional user-defined context.
The Jacobian matrix should be created in a way such that the product of 
it and the variable vector can be stored in the constraint vector.

%%% Local Variables: 
%%% mode: latex
%%% TeX-master: "manual_tex"
%%% End: 
